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This paper presents elements that form the structure of a network of data using secure stable 
and mature technologies that meet the requirement of having code free. The principle would be 
conflicting code open Tuesday where he wants to keep maximum control over the data but is 
already evidence that open source does not hide the famous backdoor possible in closed systems 
code. 

Basearemos this work experience gained in a real environment and using paravirtualization to 
show a situation more critical and now real in most companies, the virtualization of servers. 

Categories and Subject Descriptors: k.6.3 [Management of Computing and Information 
Systems]: Software Management — software selection; J. 7 [Computers in Other Sustems]: 
Command and control; 1.6.4 [Computing Methodology]: Model Validation and Analysis; 1.6.1 
[Simulation Theory]: Types of simulation; D.4.6 [Security and Protection]: Cryptographic 
controls; D.4.8 [Performance]: Measurements, Operational analysis, Simulation, Monitors 
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1. INTRODUCTION 

By working in a company which provides service to the network of municipalities 
my company is subject to any kind of attack, either via the Internet, social attacks, 
as in our own Intranet and Extranet by political enemies of our customers. Seeing 
this picture began to plan a way to keep information secure as planned and located 
most critical points in the structure, was necessary to create a map of where each 
risk manager should define on a scale of zero to ten, on'vcl criticality that the loss 
would have a certain appeal, being listed as resources to Phone onia, the network 
of data, the computers and printers documents into folders at'e fax equipment. In 
this article we point out solutions to all these points without it being necessary 
spent on purchase of software and more important with the use of technologies 
already established as stable in their specialties. 

2. METHODOLOGY 

2.1 PLATFORM OF TESTS 

We'll laboratory tests with the following equipment; 
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2 units with the following characteristics, Cpu dual core 1.6GHz, 1GB RAM, 
80 GB HD. They will be our primary and secondary servers. 

2 units with the following characteristics, 1.6 GHz Pentium CPU, 500 MB 
RAM, 40 GB HD. These units faram role of our esta is the work being 
with a Windows operating system and another with Linux Debian. 

1 Switch 8/100 Mb/s 

2.2 THE ARCHITECTURE OF THE PLATFORM OF TESTS 

2.3 POSTGRESQL 

PostgreSQL is a powerful, relational database system open source. It has more than 
15 years of activity and development of this architecture has a strong reputation 
for reliability, data integrity and accuracy. It runs on all major operating systems, 
including Linux, UNIX (AIX, BSD, HP-UX, SGI IRIX, Mac OS X, Solaris, Tru64), 
and Windows, is fully compatible ACID has the full support of foreign keys, joins, 
views, triggers and stored procedures (in go rivers languages). It includes more 
types of data SQL92 and SQL99, including INTEGER, NUMERIC, BOOLEAN, 
CHAR, VARCHAR, DATE, INTERVAL, and TIMESTAMP. It also supports stor- 
age of binary large objects, including images, sounds or video. It supports native 
programming interfaces for C / C + +, Java,. Net, Perl, Python, Ruby, Tel, ODBC, 
among others, and exceptional documentation. Why does not the Postgresql and 
Mysql? Optei for using Postgresql because it totally free and no matter the plat- 
form to be used. He has control of transactions is more mature and more stable 
and easier to restore in case of panes in hardware. 

2.4 SAMBA 

Samba is a service, used in UNIX-type operating systems, which simulates a Win- 
dows server, enabling management to be done and file sharing in a Microsoft net- 
work. In version 3, Samba are not files and provides printing services to various 
clients Windows, but can also integrate itself with Windows Server Domain, both 
as a Primary Domain Controller (PDC) or as a Domain Member. It may also be 
part of an Active Directory Domain. From recognized stability in the process of 
linking different platforms. In will have an environment with Windows and Linux 
machines working only with a source of files / data. 

2.5 NFS 

NFS - File System Network (Network File System). Protocol used to access the 
file systems on a network. It is possible to mount file systems of other machines 
through this protocol. The NFS is faster and more stable than the SAMBA but 
does not allow the interconnection between Windows and Linux without the need 
for the purchase of a software client / server to the side windos therefore only be 
used to interconnect machines with Linux. 

2.6 OPENSSH 

OpenSSH is a free version of the tools of connectivity SSH uses techniques that 
users of the Internet can trust. OpenSSH encrypting all traffic (including pass- 
words) to effectively eliminate the eavesdropping, connection of kidnappings and 
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other attacks. Moreover, provides OpenSSH tunneling and various methods of 
authentication, and supports all versions of SSH protocol. In case of connection 
between equipment and will need to prompt or graphical environment we using 
SSH, SCP for the transfer of data over the network is encrypted. 

2.7 HEARTBEAT 

The project Linux-HA (High- Availability Linux) focuses on research and implemen- 
tation of solutions for high availability (clustering) for Linux. The main component 
of this project in development is the heartbeat that works as manager of the cluster 
and its resources. As the name indicates, signalling the presence (or absence) of 
contact with the nodes of the cluster is made by sending heartbeats of small pack- 
ages addressed to all nodes in the cluster, whose confirmation of receipt by each 
node indicates the state that node. This product enters the model as a guardian of 
servers tracking any service that is necessary. In our article we are monitoring the 
services of the database, ssh, ssl, http, https. 

2.8 DRBD 

DRDB is a device designed to build blocks of clusters of high availability. This is 
done by mirroring a whole block of the device via the network. It will be responsible 
for the replication of each bit stored in the server's main Winchester 

2.9 APACHE2 

The Apache HTTP Project is a collaborative effort to develop software that aims 
to create the implementation of an HTTP server (Web) and solid open source. The 
project is managed jointly by a group of volunteers located around the world, using 
the Internet and the Web to communicate, plan and develop the server and its doc- 
umentation. This project is part of the Apache Software Foundation. In addition, 
hundreds of users contribute ideas, code and documentation for the project. As 
more robust the security point of view we are using this version. 

2.10 PHP5 

PHP (a recursive acronym for "PHP: Hypertext Preprocessor) is a programming 
language for computers interpreted, free and widely used to generate dynamic con- 
tent on the web. Despite being a language of learning and easy to use for small 
dynamic simple scripts, PHP is a powerful oriented language the objects. Despite 
being new we are using this tool as the PHP4 is not the object oriented and is no 
longer being held and that is complicating factor because we need to be not only to 
this but with all packages always updated with regard to the question less security 

2.11 SNORT 

Snort is a free software to prevent invasions of the Network (NIPS) and intrusion 
detection network (NIDS) capable of carrying out analysis of traffic in real time 
over IP networks. Snort runs of protocol analysis, content searching, and is com- 
monly used to actively or passively block a variety of attacks and crawls, such as 
buffer overflows, stealth port scans, attacks on aplicae web, tracking the SMB, and 
attempts to simulation of SO , Among other characteristics. The software is used 
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mainly for prevention of intrusion, Snort can be combined with other software, as 
SnortSnarf, sguil, OSSIM, and the Basic Analysis and Security Engine (BASE) 
to provide a visual representation of intrusion. With patches for the Snort it of- 
fers support for packet stream and virus scanning as ClamAV and with the SPADE 
abnormalities in the network can be found in layers 3 and 4 through analize history. 

2.12 IPTABLE 

It will be responsible for the blocking of services, machines and packages that are 
not allowed to travel on the network. 

2.13 XEN 

The Xen hypervisor that provides a powerful, efficient and safe for use virtualization 
for x86 CPUs, x8664, IA64, PowerPC and other architectures, has been used to 
virtualize a wide range of clients and operating systems, including Windows, Linux, 
Solaris and several versions of the BSD operating systems. It is widely regarded 
as an attractive alternative to proprietary platforms and virtualization hypervisors 
for x86 platforms and IA64. 

2.14 EXT3 

The ext3 (which means " third extended file system" ) is part of the new generation 
of management systems, the Linux file. Its biggest advantage is the support of 
journaling, which is to store information on the transactions of writing, allowing a 
rapid and reliable recovery in case of sudden interruption (for example, for lack of 
electricity). Use of this file system improves the recovery of the file system in case 
of any sudden shutdown of the computer, through sequential recording of data in 
the area of metadata and access mhash of its directory tree 

3. RESULTS OF TESTS 
3.1 STRATEGY OF TESTS 

We set up the equipment as shown in the following sections and after that start the 
testing process and cominucao using micro-specific benchmarks for this purpose. 
We chose a database and an application Postgresql testarmos PHP for the fall issue 
of reactivation of the equipment and checking time to return to normal operations, 
the rate of transfer to upgrade the base replicated, time of activation of mirror 
machine. Below enumeramos the methodologies used for testing of tolerance is 
divided into two parts and using disks or system failure in LVM, one of the machines 
failed the physical hardware and one of the servers: 
Part 1 - PHP processing. 
Part 2 - Processing of the bank Postgresql. 
failed Server 1 

Server 1 is running the virtual machines vml and vm2 
Server 2 is the virtual machines running vm3 and vm4 
Server 1 is off or has defects in operation 
Heartbeat in Server 2 detects failure of the Server 1 
Heartbeat boots virtual machines vml and vm2 in Server 2 
Server 1 is restored 
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Heartbeat in Server 1 if communicates with a Heartbeat Server 2 
Heartbeat in Server 2 paralyzes the virtual machines vml and vm2 
Heartbeat in 1 Server virtual machines vml boots and vm2 
service returns to normal 
failed Server 2 

Server 2 has vml virtual machines and vm2 

Server 1 has subsubitem Server virtual machines vm3 and vm4 

Server 2 is switched off or has defects in operation 

Heartbeat in Server 1 detects failure of Server 2 

Heartbeat boots virtual machines vm3 and vm4 in Server 1 

Server 2 is restored 

Heartbeat in Server 2 would communicate with Heartbeat in Server 1 
Heartbeat in Server 1 paralyzes the virtual machines vm3 and vm4 
Heartbeat Server 2 boots in virtual machines vm3 and vm4 
service returns to normal 
These tests were failures of tolerance will be made as follows: Simulation of the 
failure of the server by stopping the service of heartbeat 

Simulating the failure of the server 1, enter the following command in the server 
1: /ctc/init.d/heartbeat stop 

Stop the server through its forced shutdown (pulling power cord from the) Stop 
the server through its disengagement correct, (command 'shutdown') 

3.2 MOUNTING CONFIGURATION AND THE ENVIRONMENT 

Installation of the Linux operating system Debian Etch. On devices defined as 
servers. We will be using this distribution by the stable version available on the date 
of creation of this article. In this installation use partitioning EXT3 for installation 
of data, and the division of HD in our area of 2.7 GB to SWAP and the rest of the 
unit for data. 

Instalation NFS. sudo aptitude install nfs-common nfs-server-kernel portmap 
Once installed the packages edit /etc/exports and add the directories to be ac- 
cessed remotely, see the example below: 
/u/usr 10.0.2.6 (rw, sync) 

Above are sharing the directory /u/ usr only to the machine 10.0.2.6 allowing 
this writing and reading and forcing syncronismo between the two machines. 

Installing SAMBA, sudo aptitude install smbfs samba samba-common smbclient 

Edit /etc/samba/smb.conf and observe the following parameters: 

workgroup = XXXXXXXX 

server string = XXXXXXXX 

printcap name = /ctc/printcap 

load printers = no 

socket options = TCP_NODELAY SO_RCVBUF=8192 SO.SNDBUF=8192 

dns proxy = no 

netbios name = padrao 

netbios aliases = padrao 

map to guest = never 

os level = 99 
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preferred master = no 
domain master = no 
wins support = no 
dead time = 
domain logons = no 
printcap name = cups 
printing = cups 

log file = /var/log/samba/log.%m 

max log size = 50 

debug level = 1 

security = share 

unix password sync = yes 

password level = 

null passwords = yes 

encrypt passwords = true 

smb passwd file = /etc/samba/smbpasswd 

username map = /etc/samba/smbusers 

username level = 8 

add machine script = /usr/sbin/adduscr -n -r -g machines -c "Samba machine" 
-d /dev/null -s /bin/false %u 
passdb backend = smbpasswd 
idmap uid = 16777216-33554431 
idmap gid = f67772f 6-33554431 
template shell = /bin/false 
winbind use default domain = no 
bind interfaces only = no 
hide dot files = no 
[Dcsenv] 

comment = XXXXXXXXXXXXXXXx 

path = /XXXXXXXXXX 

public = no 

browseable = yes 

guest only = no 

guest ok = yes 

writable = yes 

preserve case = No 

short preserve case = No 

directory mask = 0777 

valid users = luciana 

create mask = 0777 

available = yes 

Installing DRDB. The advantage of DRDB8 on SRDB7 are: It allows resources 
to be master of both the time and can be mounted with Permissions of reading 
and writing. Now we will compile the modules from DRDB8 to be loaded into the 
kernel. For this we need the packages build-essential and kernel-headers-xen. Do 
into the prompt; 
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sudo aptitude install drbd8-utils drbd8-module-source drbd8-source build-essential 
linux-headers-xen sudo sudo m-a-i-module drbd8-source sudo update-modules sudo 
modprobc drbd 

This will compile the modules for kernel / drivers / block / drbd.ko and will be 
used for this kernel. A configuration pado was set up in / etc / drbd.conf 
Configuration: 
Edit o /etc/drbd.conf 
global usage-count yes; 
common syncer rate 10M; 

resource rO protocol C; handlers pri-on-incon-degr "echo o l /proc/sysrq-trigger ; 
halt -f" ; pri-lost-after-sb "echo o l /proc/sysrq-trigger ; halt -f ; local-io-error "echo 
o i /proc/sysrq-trigger ; halt -f; outdate-peer "/usr/sbin/drbd-peer-outdater"; 

startup 

disk on-io-error detach; 

net allow-two-primaries; after-sb-Opri disconnect; after-sb-lpri disconnect; after- 
sb-2pri disconnect; rr-conflict disconnect; 
syncer rate 10M; al-extents 257; 

onnodef device /dev/drbdO; disk /dev/sda3; address 192.168.0.128:7788; flexible- 
meta-disk internal; 

on nodc2 device /dev/drbdO; disk /dev/sda3; address 192.168.0.129:7788; meta- 
disk internal; 

"Allow-two-primaries" option that allows you to be mounted as master "master" 
at the beginning of our network. Copy /etc/drbd.conf for o node 2 and restart 
drbd with the following command, sudo / init.d / drbd restart 

If you want to check the state run the command below 

sudo /etc/init.d/drbd status 

This should be the response if everything is OK. 

drbd driver loaded OK; device status: 

version: 8.0.3 (api:86/proto:86) 

SVN Revision: 2881 build by root@nodel, 2008-01-20 12:48:36 0: cs:Connected 
st:Sccondary/Secondary ds:UpToDate/UpToDate C r— ns: 143004 nr:0 dw:0 dr: 143004 
al:0 bm:43 lo:0 pe:0 ua:0 ap:0 resync: used:0/31 hits:8916 misses:22 starving:0 
dirty:0 changed:22 actdog: used:0/257 hits:0 misses:0 starving:0 dirty:0 changed:0 

replace the appeal to the master with the following command in equipment 

sudo drbdadm primary rO 

and check the status again 

sudo /etc/init.d/drbd status 

drbd driver loaded OK; device status: 

version: 8.0.3 (api:86/proto:86) 

SVN Revision: 2881 build by root@nodcl, 2008-01-20 12:48:36 0: cs:Connected 
st:Primary/Primary ds:UpToDate/UpToDate C r— ns:143004 nr:0 dw:0 dr:143004 
al:0 bm:43 lo:0 pe:0 ua:0 ap:0 resync: used:0/31 hits:8916 misses:22 starving:0 
dirty:0 changed:22 actdog: used:0/257 hits:0 misses:0 starving:0 dirty:0 changed:0 

As you can see action is "master" in both of us device. And the drbd is now 
accessible on / dev/drbdO 

File system 
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We can now create the file system in / dev/drbdO with the following command 
sudo mkfs.ocfs2 /dev/drbdO 

This can be mounted simultaneously in both with the commands below: 

sudo mkdir /drbdO 

sudo mount. ocfs2 /dev/drbdO /drbdO 

Now we have a syncronismo between storage devices. 

Init script 

We have to make sure that, after restart, the system will set drbd resources 
again to " master" and mount a " / drbdO" before starting the Heartbeat and Xen 
machines. 

Edit /etc/init.d/mountdrbd.sh 

drbdadm primary rO 

mount. ocfs2 /dev/drbdO /mnt 

make a symbolic link to executable / etc/rc3.d/S99mountdrbd.sh 
sudo chmode +x /etc/init.d/mountdrbd.sh 

sudo In -s /etc/init.d/mountdrbd.sh /etc/rc3.d/S99mountdrbd.sh 
In fact, this step can also be integrated to Heartbeat, adding adequate resources 
for the setting. But as time is that vai do with this script. 

Installation Heartbeat2. Now we can install and configure the Heartbeat 2 

sudo apt-get install heartbeat-2 heartbeat-2-gui 

Edit /etc/ha.d/ha.cf 

crm on 

beast ethO 

node nodel node2 

restart heartbeat2 com 

sudo /etc/init.d/heartbeat restart 

3.3 Startup 

Edit the file /root /cluster/bootstrap. xml 
cluster .property _set id=" bootstrap" 
attributes 

nvpair id="bootstrap01" namc=" transition- idle-timeout" value="60"/ 

nvpair id=" bootstrap02" namc=" dcfault-resource-stickiness" value=" INFINITY" / 

nvpair id="bootstrap03" name=" default-resource- failure-stickiness" value="-500"/ 

nvpair id="bootstrap04" namc=" stonith-enablcd" value="true"/ 

nvpair id="bootstrap05" namc=" stonith- action" value=" reboot"/ 

nvpair id="bootstrap06" name=" symmetric-cluster" value="true"/ 

nvpair id=" bootstrap07" name=" no-quorum-policy" value=" stop" / 

nvpair id=" bootstrap08" name=" stop-orphan-resources" value=" true" / 

nvpair id=" bootstrap09" name=" stop-orphan-actions" value=" true" / 

nvpair id="bootstraplO" namc=" is-managed-default" valuc="truc"/ 

/ attributes 

/ cluster .property _set 

Load the file with the following command 

sudo cibadmin -C crm_config -x /root/cluster/bootstrap.xml 

This will start the Cluster with the values set in xml file 
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Setting up the device STONITH 

Using the command below the keys to create trust between the servers, 
sudo ssh-keygen 

- save key under /root/.ssh/* 

- dont give any passphrase 

scp /root/. ssh/id_rsa. pub node2:/root/.ssh/authorized_keys 
Now make sure you can log on the server 2 from the server 1 without using 
password. 

sudo ssh -q -x -n -1 root "node2" "Is -la" 
Stonith of configuring the server 2 
/ root / cluster / stonith. xml 

clone id=" stonithclonc" globally _uniquc=" false" 
instance_attributes id=" stonithclone" 
attributes 

nvpair id="stonithclonc01" name=" clone_node_max" value="l"/ 

/ attributes 

instance_attributes 

primitive id=" stonithclone" class=" stonith" type="external/ssh" provider=" heartbeat" 
operations 

op name=" monitor" interval^" 5s" timeout="20s" prereq=" nothing" id=" stonithclone- 
opOl"/ 

op namc=" start" timeout=" 20s" prereq=" nothing" id="stonithclone-op02"/ 
/operations 

instance_attributes id=" stonithclonc" 
attributes 

nvpair id="stonithclone01" name=" hostlist" value="nodel,node2"/ 

/ attributes 

/instance_attributes 

/primitive 

/ clone 

Load with the following command 

sudo cibadmin -C -o resources -x /root/cluster/stonith.xml 

Xen the cluster resources. Now we can add the virtual machine XEN in the 
cluster. 

Now we can add to the Xen virtual machine cluster resource. Lets say that we 
have a Xen to view the machine called vmOl. The cofigurao and image files to keep 
us in vmOl /drbd0/xen/vm01/ in vmOl.cfg and vmOl-diskO.img respectively. 

Edit /root/cluster/vmOl.xml 

resources 

primitive id="vm01" class="ocf" type="Xen" provider=" heartbeat" 
operations 

op id="vm01-op01" name=" monitor" interval=" 10s" timeout="60s" prereq=" nothing" / 
op id="vm01-op02" name=" start" timcout="60s" start_delay=" 0" / 
op id="vm01-op03" name="stop" timeout=" 300s" / 
/operations 

instancc_attributcs id="vm01" 
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attributes 

nvpair id=" vmOl-attrOl" name="xmfile" value="/drbdO/xen/vm01/vm01.cfg"/ 

nvpair id=" vm01-attr02" name=" target_role" value=" started" / 

/ attributes 

/ instance_attributes 

meta_attributes id=" vmOl-metaOl" 

attributes 

nvpair id=" vmOl-meta-attrOl" name=" allow .migrate" value=" true" / 

/attributes 

/ meta_attributes 

/primitive 

/resources 

Load this file with the following command. 

sudo cibadmin -C -o resources -x /root /cluster /vmOl.xml 

Tracking tools. With the command "crm _mon" you can track the inclusion of 
resources and in the cluster, 
sudo crm_mon Refresh in 14s... 
The result of this command will be: 



Last updated: Fri Jan 25 17:26:10 2008 

Current DC: node2 (83972cf7-0b56-4299-8e42-69b3411377a7) 
2 Nodes configured. 
6 Resources configured. 



Node: node2 (83972cf7-0b56-4299-8e42-69b3411377a7): online 
Node: nodel (6bfd2aa7-bl32-4104-913c-c34ef03a4dba): online 
Clone Set: stonithclonc 

stonithclone:0 (stonith:external/ssh): Started nodel 
stonithclone:l (stonith:external/ssh): Started node2 
vmOl (heartbeat: :ocf:Xen): Started node2 

There is also a GUI available (graphical tool). To use it just set a password for 
the user "hacluster" with the following command and run the command "hb _gui" 
sudo passwd hacluster 
password 
re type password 
sudo hb_gui & 

4. ANALYSIS OF RESULTS 

The fact work with LVM facilitated the mirroring of machines but the total security 
was not achieved because when we have to save the records in the course mirroring 
lost the last record and the bank needed to make the rollback. But the resumption 
of service in the event of the fall of the main server was made in seconds not creating 
inconvenience to users of the network. 

Although the performance was higher with the use of XEN no details in this 
article because this item is not the purpose of it. 
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5. CONCLUSION 

Looking up the sequence of servers rcligamento of the structure is made entirely 
stable and secure even in tests in Part 1 where only in processing memory was being 
implemented in PHP in the second machine (Server 2), after the fall continued 
smoothly. 

In practical terms only at the end of the business can religar the main server 
(Server 1) again because of the time synchronization between the two high and for 
implying in the network for several minutes. 
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